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Key findings 





This study examined data from the 2007/08 grade 9 cohort in four Oregon 
districts to find early warning indicators of students who may drop out or fail 
to graduate from high school on time. The study identified four indicators that 
provided valuable early warning signals about students who did not graduate 
on time, particularly about students who dropped out of high school: 

Grade 8 attendance rate below 80 percent. 

Grade 9 attendance rate below 80 percent. 

Grade 8 grade point average (GPA) below 2.0. 

Grade 9 GPA below 2.0. 

When the influence of demographic, achievement, and behavioral characteristics 
and differences in the schools that students attended were considered at the 
same time, only gender, English learner student status, and attendance and 
GPA in grades 8 and 9 were associated with graduation outcomes. 
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Summary 


A substantial body of recent research has attempted to identify the characteristics of stu- 
dents who graduate from high school on time and of those who do not. One motivation for 
this research has been the desire to develop indicators that can provide early identification 
of students who might not complete high school on time so that educators can design 
interventions to get these students back on track for graduation. 

This study looked at the graduation outcomes of a cohort of students who began grade 9 
in the 2007/08 school year in four Oregon districts. It analyzed factors related to three key 
graduation outcomes: dropping out within four years of entering grade 9, staying in school 
but not graduating on time (within four years), and graduating on time. 

The results call particular attention to male students and to English learner students 
because these students had lower on-time graduation rates than other students. In addi- 
tion, the study confirms findings from previous research that high attendance and grade 
point average (GPA) in grades 8 and 9 are significantly associated with graduating on time. 
After other factors were accounted for, race/ethnicity and achievement on standardized 
tests were less predictive of graduating on time. Unlike previous studies, this one did not 
find that, after accounting for other factors, discipline in grade 9 was substantially associat- 
ed with graduating on time. 

The study has three key findings: 

• Four indicators — grade 8 attendance rate below 80 percent, grade 9 attendance 
rate below 80 percent, grade 8 GPA below 2.0, and grade 9 GPA below 2.0 — pro- 
vided valuable early warning signals about students who did not graduate on time. 

• When other factors were not accounted for, students’ high school graduation out- 
comes differed according to their demographic characteristics, special education 
status, English learner student status, attendance and achievement in grades 8 and 
9, and behavior (suspension or expulsion) in grade 9. 

• However, when the influence of demographic, achievement, and behavioral char- 
acteristics and differences in the schools that students attended were considered 
at the same time, only gender, English learner student status, and attendance and 
GPA in grades 8 and 9 were associated with graduation outcomes. In addition, 
differences in graduation outcome by gender and English learner student status 
diminished for students with higher GPAs in grade 9. 

In this study, attendance and GPA in grades 8 and 9 were the most predictive indicators 
of graduation outcome. And unlike gender or English learner student status, they can be 
influenced by teachers and parents to help keep students on track for graduation. 
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Why this study? 


Regional Educational Laboratory (REL) Northwest undertook this study for two reasons: 
to add to the growing body of research about characteristics of students who fail to com- 
plete high school on time and to provide information to the Oregon Leadership Network 
concerning students who graduated from high school on time and those who did not. 
The Oregon Leadership Network is an alliance of 16 districts, two education service dis- 
tricts, and several community groups that have worked together for more than a decade to 
promote successful outcomes for all students (Education Northwest, 2011). Improving grad- 
uation rates is a high priority for the alliance districts. This study informs work under way 
in the alliance districts about developing early warning systems that can identify students 
who are likely to need additional support to graduate on time. 

Dropout and on-time graduation rates in Oregon 

Overall, dropout rates for Oregon students have declined in recent years. However, the 
dropout rate varies substantially for students of different races/ethnicities. In 2010/11, 
3.3 percent of Oregon’s high school students dropped out (Oregon Department of Educa- 
tion, 2012). 1 Statewide, the dropout rate for Black students and American Indian students 
was twice the rate for White students, while the dropout rate for Hispanic students was 1.6 
times the rate for White students (Oregon Department of Education, 2012). 

Across the four study districts, 7 percent of students who began grade 9 in 2007/08 dropped 
out. The overall dropout rate for racial/ethnic minority students was 1.7 times higher 
than the rate for White students. Within districts, dropout rates ranged from 4 percent to 
10 percent for White students and from 6 percent to 15 percent for racial/ethnic minority 
students (see table A2 in appendix A). 

Across all Oregon districts, 68 percent of students who began grade 9 in 2007/08 gradu- 
ated on time (within four years; Oregon Department of Education, 2013). This included 
52 percent of American Indian students, 54 percent of Black students, 58 percent of His- 
panic students, 70 percent of White students, and 78 percent of Asian students. 

Across the study districts, 73 percent of students who began grade 9 in 2007/08 graduated 
on time. Within the study districts, on-time graduation rates ranged from 59 percent to 
77 percent. Overall, 77 percent of White students and 67 percent of racial/ethnic minority 
students graduated on time. By individual district, on-time graduation rates ranged from 
68 percent to 81 percent for White students and from 50 percent to 72 percent for racial / 
ethnic minority students. 

Students who drop out of high school often suffer severe economic consequences. For 
example, compared with individuals with a high school diploma, dropouts are less likely to 
be employed, and when employed, they work fewer weeks per year. In addition, dropouts 
earn half as much as individuals with a diploma and are half as likely to be covered by a 
health plan provided through a union or an employer (Rouse, 2007). Recent studies show 
that the earnings gap between high school dropouts and high school graduates is growing. 
Overall, the economic consequences of dropping out are increasingly grim (Heckman, 
Humphries, & Mader, 2011). Furthermore, dropping out is associated with disadvantages 
in areas of life beyond employment and wages. For example, dropouts commit more crimes, 
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suffer poorer health, and are less likely to vote and engage constructively in their commu- 
nities than people who finish high school (Pleis, Ward, & Lucas, 2010; Rumberger, 2011). 


The individual costs of dropping out are mirrored by the substantial costs to society, 
including approximately $240,000 in lost tax revenue over the lifetime of an average 
dropout and higher expenses associated with increased crime, welfare, and health care 
(Chapman, Laird, Ifill, & KewalRamani, 2011; Rumberger, 2011). Some dropouts go on to 
earn a General Educational Development certificate (GED). However, evidence suggests 
that the GED is not a satisfactory substitute for completing high school. More specifical- 
ly, the GED does not improve economic outcomes for most of those who attain it, and 
holding a GED does little to improve chances for success in postsecondary institutions 
(Heckman et al., 2011). 

Oregon Leadership Network districts, like many other districts across the country, are 
investigating ways to identify early those students who may not graduate from high school 
on time. This study addresses this need by examining how high school graduation out- 
comes are associated with gender, race/ethnicity, special education status, English learner 
student status, attendance and achievement in grades 8 and 9, and suspension or expuL 
sion in grade 9, after controlling for each of the remaining variables. 2 Finding out which 
variables are predictive of dropping out or failing to graduate within four years can help 
educators develop better early warning indicators to monitor and intervene on behalf of 
students who need assistance. 

Connections to previous research 
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Research on student motivation (Usher & Kober, 2012) and the engagement of high 
school students with their schools (National Research Council, 2004) suggests that stu- 
dents’ persistence in school is determined by a complex interplay of behavioral, cogni- 
tive, social, and emotional factors that operate within students, families, classrooms, and 
schools, as well as at other levels. At the same time, a substantial body of research has 
investigated characteristics of middle school students and students in grade 9 that provid- 
ed early warning of students who eventually dropped out of high school. Key points from 
this research are summarized in box 1. Findings from representative studies are summa- 
rized in appendix B. 


Box 1. Previous research findings about students who did not graduate from high 
school on time 

Students’ attendance, achievement, and behavior have stronger relationships with graduating 
than do their race/ethnicity and their achievement on state tests in reading and math. 

Students who drop out of high school can be identified by examining their attendance, achieve- 
ment, and behavior in grade 9. 

Attendance and achievement in the middle grades are associated with high school graduation 
outcomes, and these factors can be used to identify students who are likely to drop out of high 
school. 

Source: Summarized from sources described in appendix B and table Bl. 
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This study is similar to previous research in two ways. First, analyses include student demo- 
graphic characteristics, attendance, and behavior along with course grades and results of 
standardized tests, which previous research has found to be related to graduation outcomes. 
Second, the study examined thresholds of attendance and achievement in grades 8 and 9 
that identified students who did not graduate from high school on time. 

What the study examined 


This study examined three types of graduation outcomes: dropping out within four years of 
entering grade 9, staying in school but not graduating on time (within four years of enter- 
ing grade 9), and graduating on time. Each outcome was addressed by a research question: 

1. Characterizing students with different graduation outcomes. Did students’ high 
school graduation outcomes differ by their gender, race/ethnicity, special education 
status, English learner student status, attendance and achievement in grades 8 and 9, 
and behavior (suspension or expulsion) in grade 9? 

2. Examining statistical relationships among the predictors. How were high school 
graduation outcomes associated with each of the above variables, after the study con- 
trolled for the other variables and for characteristics (both observed and unobserved) 
of the schools that students attended? 

3. Identifying indicators of failing to graduate on time. What levels of attendance and 
achievement in grades 8 and 9 and suspension and expulsion in grade 9 could provide 
early identification of students who might not graduate on time? 

A brief description of the study sample and analytic methods is given in box 2. Additional 
information about the outcomes and analytic methods is provided in table A1 in appendix 
A and in appendix C. 


Box 2. Data and methods 

The study sample is a cohort of 6,118 students who entered grade 9 in four Oregon school dis- 
tricts in 2007/08. All data for the study came from state and district databases. The Oregon 
Department of Education provided data for enrollment, demographics, suspension or expul- 
sion, participation in special programs, and graduation and dropout status. The four districts 
provided grade point average (GPA) data. In the database, 276 students in the four districts 
who earned a General Educational Development certificate were recorded as nondropouts and 
nongraduates and thus were not included in the study. Students who graduated after making 
up credits after their senior year ended were treated as on-time graduates if they were record- 
ed as graduates in the database for the 2010/11 school year (September 1, 2010, to August 
31, 2011). 

The high school graduation outcomes examined in the study are “dropped out,” “stayed 
in school but did not graduate on time,” and “graduated on time." “Dropped out” refers to 
students who dropped out within four years of entering grade 9. “Stayed in school but did not 
graduate on time” refers to students who did not drop out within four years of entering grade 9 
but did not graduate within four years. “Graduated on time” refers to students who graduated 

(continued) 
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Box 2. Data and methods (continued) 


within four years of entering grade 9. Additional information about the variables used in the 
study is given in table A1 in appendix A. 

Percentages of students who dropped out, stayed in school but did not graduate on time, 
and graduated on time were calculated and reported by gender, race/ethnicity, special educa- 
tion status, and English learner student status. Average levels of attendance, GPA, achieve- 
ment on state tests in reading and math, and suspensions or expulsions were calculated and 
reported for students with each graduation outcome. In addition, percentages of students with 
each graduation outcome were calculated and reported for students with different levels of 
attendance, GPA, achievement on state tests, and suspension or expulsion. 

Regression analysis was used to describe how gender, race/ethnicity, special education 
status, English learner student status, attendance in grades 8 and 9, GPA in grades 8 and 9, 
achievement on state tests in grade 8, and suspension or expulsion in grade 9 together were 
associated with graduation outcomes. These variables were chosen because research shows 
that student attendance, achievement, and behavior in grades 8 and 9 are associated with 
graduating and dropping out and because the districts requested an analysis of how these 
variables were associated with graduation outcomes for their population of students. 

The study controlled for differences in the schools that students attended by including 
fixed effects for schools in the regression model. Models without school fixed effects produce 
estimates that combine what is happening both within and between schools. Including fixed 
effects produces estimates based only on students’ experiences within their school. Using 
fixed effects to control for school characteristics (both observed and unobserved) makes the 
results more relevant for school staff members who are deciding which of their students will 
get dropout prevention services. Decisions at this level require school staff members to think 
about what is going on within their school rather than think about whether some of their stu- 
dents might benefit from attending other schools. 

Additional information about the analytic methods is provided in appendix C. 
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What the study found 


Achievement and attendance in grades 8 and 9 provide strong early warning signals about 
students who may need additional support to graduate on time. More specifically, the study 
found that attendance in grade 8 or 9 of less than 80 percent and a GPA in grade 8 or 9 of 
less than 2.0 provided early identification of students who did not graduate on time. The 
study also identified differences in rates of graduating within four years of entering grade 
9 by students’ gender, race/ethnicity, special education status, and English learner student 
status. However, when the influence of demographic, achievement, and behavioral charac- 
teristics and differences in the schools that students attended were considered at the same 
time, only gender, English learner student status, and attendance and GPA in grades 8 and 
9 were associated with graduation outcomes. 

These results are consistent with other research, which has found that on-time graduation 
rates differ for students with different characteristics and that attendance and achieve- 
ment in grades 8 and 9 have stronger associations with graduation outcomes than do 
race/ethnicity, special education status, English learner students status, and results of stan- 
dardized tests of achievement in reading and math. The study differs from other research 
in finding that discipline in grade 9 was not substantially associated with graduating on 
time after accounting for other factors. 
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When other factors were not accounted for, students’ high school graduation outcomes differed 
according to their demographic characteristics, special education status, English learner student 
status, attendance and achievement in grades 8 and 9, and behavior in grade 9 


Overall, 7 percent of the students who entered grade 9 in the four study districts in 2007/08 
dropped out within four years, 20 percent stayed in school but did not graduate on time, 
and 73 percent graduated on time. Dropout rates ranged from 5 to 12 percent across the 
districts, while on-time graduation rates ranged from 59 to 77 percent (figure 1). 


High school outcomes varied across the cohort by student gender, race/ethnicity, special 
education status, and English learner student status (table 1). 

• Male students dropped out at a rate 1.6 times higher than female students and 
stayed in school but did not graduate on time at a rate 1.4 times higher than female 
students did. 

• Except for Asian students, racial/ethnic minority students dropped out and stayed 
in school but did not graduate on time at higher rates than White students did. 

• Students in special education dropped out or stayed in school but did not graduate 
on time at rates approximately twice as high as those for students not in special 
education. 

• English learner students dropped out at rates more than twice as high as those for 
non-English learner students and stayed in school but did not graduate on time at 
rates 1.5 times higher. 


Overall, 7 percent 
of the students 
who entered 
grade 9 in the four 
study districts in 
2007/08 dropped 
out within four 
years, 20 percent 
stayed in school 
but did not 
graduate on time, 
and 73 percent 
graduated on time 


Figure 1. In the 2007/08 grade 9 cohort in four Oregon districts, 7 percent of 
students dropped out, 20 percent did not graduate on time, and 73 percent 
graduated on time, with some variation among districts 


Percent 



District A District B District C District D All districts 

(n = 2,596) (n = 1,750) (n = 1,173) (n = 599) (n = 6,118) 


Note: Dropped out refers to students who dropped out within four years of entering grade 9. Stayed in school 
but did not graduate on time refers to students who did not drop out within four years of entering grade 9 but 
did not graduate within four years. Graduated on time refers to students who graduated within four years of 
entering grade 9. 


Source: Author’s analysis of data from Oregon state and district databases; see box 2 for details. 
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Table 1. High school outcomes by student demographic characteristics, program participation, and 
behavior, grade 9 cohort of 2007/08 



Dropped out 

Stayed in school but did 
not graduate on time 

Graduated on time 

Characteristic 

Number 

Percent 

Rate ratio 9 

Percent 

Rate ratio 9 

Percent 

Rate ratio 9 

Total 

6,118 

7 

na 

20 

na 

73 

na 

Gender 

Male 

3,159 

8 

1.6 

23 

1.4 

69 

0.9 

Female 

2,959 

5 

na 

17 

na 

78 

na 

Race/ethnicity 

American Indian 

60 

15 

3.0 

30 

1.7 

55 

0.7 

Asian 

685 

3 

0.6 

12 

0.7 

86 

1.1 

Black 

325 

8 

1.6 

33 

1.8 

59 

0.8 

Hispanic 

908 

14 

2.8 

32 

1.8 

55 

0.7 

More than one race/other 

226 

10 

2.0 

20 

1.1 

70 

0.9 

White 

3,914 

5 

na 

18 

na 

77 

na 

Special education status 

In special education 

178 

10 

2.0 

36 

1.9 

54 

0.7 

Not in special education 

5,940 

5 

na 

19 

na 

74 

na 

English learner student status 

English learner student 

614 

14 

2.3 

29 

1.5 

57 

0.8 

Non-English learner student 

5,504 

6 

na 

19 

na 

75 

na 

Behavior 

Suspended or expelled grade 9 

564 

21 

4.2 

46 

2.7 

33 

0.4 

Not suspended or expelled in grade 9 

5,554 

5 

na 

17 

na 

78 

na 


na is not applicable because the group is the reference group. 

Note: Dropped out refers to students who dropped out within four years of entering grade 9. Stayed in school but did not graduate on 
time refers to students who did not drop out within four years of entering grade 9 but did not graduate within four years. Graduated 
on time refers to students who graduated within four years of entering grade 9. The combined percentages of the three outcomes — 
dropped out, stayed in school but did not graduate on time, and graduated on time — may not sum to 100 because of rounding. 

a. Compares percentages for a target group, for example, male students, against percentages for a reference group, for example, 
female students. Reference groups are the last group in each section. For example, the rate ratio of 1.6 for male students under the 
percent dropped out column indicates that the percentage of male students (target group) who dropped out was 1.6 times higher than 
the percentage of female students (reference group) who dropped out. 

Source: Author’s analysis of data from Oregon state and district databases; see box 2 for details. 


Graduation outcomes by student demographic characteristics and program participation 
are presented by district in table A2 in appendix A. 

For the whole cohort of students and within each district, average levels of attendance, 
achievement, and behavior were substantially different among students who dropped out, 
stayed in school but did not graduate on time, and graduated on time (table 2; see also 
table A3 in appendix A). Students who dropped out had the lowest average attendance 
and GPA, lowest rates of scoring proficient on state tests in reading and math, and highest 
rates of suspension and expulsion. Students who graduated on time had the highest average 
attendance rates, GPA, and rates of scoring proficient on state tests in reading and math 
and the lowest rates of suspension and expulsion. Students who stayed in school but did 
not graduate on time had attendance rates, GPAs, rates of scoring proficient on state tests, 
and suspension and expulsion between the levels for dropouts and on-time graduates. 
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Table 2. Average attendance, achievement, and suspension or expulsion in grades 
8 and 9 by graduation outcome, grade 9 cohort of 2007/08 


Grade and variable 

Dropped out 
(n = 403) 

Stayed in school but did 
not graduate on time 
(n = 1,216) 

Graduated 
on time 
(n = 4,499) 

Grade 8 

Attendance (percent) 

87 

91 

95 

Grade point average 

2.0 

2.3 

3.2 

Proficient on state reading test (percent) 

45 

60 

81 

Proficient on state math test (percent) 

51 

64 

85 

Grade 9 

Attendance (percent) 

80 

87 

95 

Grade point average 

1.3 

1.7 

3.1 

Suspended or expelled (percent) 

29 

21 

4 


Note: Dropped out refers to students who dropped out within four years of entering grade 9. Stayed in school 
but did not graduate on time refers to students who did not drop out within four years of entering grade 9 but 
did not graduate within four years. Graduated on time refers to students who graduated within four years of 
entering grade 9. State tests in reading and math were only at grade 8. Percent proficient includes students 
meeting or exceeding the standard for proficiency. Suspension and expulsion data were not available for 
grade 8. 

Source: Author's analysis of data from Oregon state and district databases; see box 2 for details. 


When the influence of demographic, achievement, and behavioral characteristics and differences in 
the schools that students attended were considered at the same time, only gender, English learner 
student status, and attendance and grade point average in grades 8 and 9 were associated with 
graduation outcomes 

The second research question addresses how students’ graduation status four years after 
entering grade 9 was associated with their gender, race/ethnicity, special education status, 
English learner student status, attendance and achievement in grades 8 and 9, and suspen- 
sion or expulsion in grade 9 when all these factors were considered at the same time. This 
was done through regression modeling. Details of the modeling are given in appendix C. 


After other factors 
were accounted 
for, students’ 
graduation 
outcomes four 
years after 
entering grade 9 
were associated 
with their gender, 
English learner 
student status, 
attendance 
in grade 8, 
attendance in 
grade 9, GPA 
in grade 8, and 
GPA in grade 9 


After other factors were accounted for, students’ graduation outcomes four years after 
entering grade 9 were associated with their gender, English learner student status, atten- 
dance in grade 8, attendance in grade 9, GPA in grade 8, and GPA in grade 9. To illustrate 
the association of these variables with graduating four years after entering grade 9, proba- 
bilities of graduating four years after entering grade 9 were calculated from the model for 
values of each of these variables (table 3). 


At higher grade 9 GPAs, differences in on-time graduation rates by gender shrank (figure 2). 

Similarly, differences in on-time graduation rates between English learner students and 
other students shrank at higher grade 9 GPAs (figure 3). 

In contrast to the reduced differences in on-time graduation rates between male and female 
students and between English learner and non-English learner students observed at higher 
grade 9 GPAs (see figures 2 and 3), differences between these groups of students remained 
constant across attendance in grades 8 and 9 and grade 8 GPAs. 
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Table 3. Estimated probabilities of graduating within four years of entering grade 9, 
by attendance and grade point average in grades 8 and 9 and gender and English 
learner student status, grade 9 cohort of 2007/08 


Variable and value 

Estimated probability of 
graduating on time 

Rate ratio 

Grade 8 attendance 

95 percent 

.74 

1.12 

85 percent 

.71 

1.08 

75 percent 

.69 

1.05 

65 percent 

.66 

na 

Grade 9 attendance 

95 percent 

.75 

1.21 

85 percent 

.71 

1.15 

75 percent 

.67 

1.08 

65 percent 

.62 

na 

Grade 8 grade point average 

4.0 

.78 

1.13 

3.0 

.75 

1.09 

2.0 

.72 

1.04 

1.0 

.69 

na 

Grade 9 grade point average 

4.0 

.92 

2.00 

3.0 

.83 

1.80 

2.0 

.67 

1.46 

1.0 

.46 

na 

Female 

.76 

1.06 

Male 

.72 

na 

Non-English learner student 

.74 

1.07 

English learner student 

.69 

na 


na is not applicable because the group is the reference group. 

Note: Estimated probabilities were calculated from the regression model described in appendix C. 
Source: Author's analysis of data from Oregon state and district databases; see box 2 for details. 


Four indicators-grade 8 attendance rate below 80 percent, grade 9 attendance rate below 
80 percent, grade 8 GPA below 2.0, and grade 9 GPA below 2.0-provided valuable early warning 
signals about students who did not graduate on time, particularly about students who dropped out 

The third research question addresses whether indicators could be developed to provide 
early identification of students who might not graduate on time. Students identified by such 
indicators could be provided with programs and resources to improve their attendance and 
achievement and thus increase their likelihood of graduating on time. Regression mod- 
eling showed that when students’ gender, race/ethnicity, and program participation were 
considered together with their attendance, achievement, and behavior in grades 8 and 
9, the following indicators were not related to graduation outcomes in a statistically sig- 
nificant way: race/ethnicity, special education status, meeting standards on state tests in 
reading and math, and suspension or expulsion in grade 9 (see table Cl in appendix C). 

The following variables were associated with graduation outcomes: gender, English learner 
student status, attendance in grade 8, attendance in grade 9, GPA in grade 8, and GPA in 
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Figure 2. At higher grade point averages, differences in on-time graduation rates 
shrank between male and female students, grade 9 cohort of 2007/08 


Estimated probability 



Grade 9 grade point average 

Note: Estimated probabilities were calculated from the regression model described in appendix C. 
Source: Author's analysis of data from Oregon state and district databases; see box 2 for details. 


Figure 3. At higher grade point averages, differences in on-time graduation rates 
shrank between English learner students and other students, grade 9 cohort of 
2007/08 


Estimated probability 



Grade 9 grade point average 

Note: Estimated probabilities were calculated from the regression model described in appendix C. 
Source: Author's analysis of data from Oregon state and district databases; see box 2 for details. 
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grade 9 (see table Cl in appendix C). While gender and English learner student status were 
associated with graduation outcomes, these variables were not considered as early warning 
indicators for two reasons. First, students, parents, and teachers cannot control these vari- 
ables in the way they can control students’ attendance and achievement. Second, using 
gender and English learner student status in an early warning system might encourage 
stereotyping of certain groups of students. In contrast, attendance and GPA in grades 8 
and 9 are directly influenced by students, parents, and teachers. As a result, these four vari- 
ables were selected as indicators. The approach used in this study follows Allensworth and 
Easton (2005, 2007) in focusing on academic variables as early warning indicators. 

Proportions of students who graduated or did not graduate within four years of entering 
grade 9 at different rates of attendance and levels of GPA are displayed in table 4. For 
brevity, data are aggregated for attendance below 80 percent and GPA below 2.0. 

To be useful, early warning indicators must meet two criteria. First, they must identify sub- 
stantial proportions of students who failed to graduate on time. Second, they must identify 
relatively few students who graduated on time (Pepe, 2003). In this sample, attendance of 


Table 4. High school outcomes at different rates of attendance and levels of GPA in 

grades 8 and 9, grade 9 cohort of 2007/08 








Percent who 


Attendance 




stayed in school 

Percent who 

and GPA 


Number of 

Percent who 

but did not 

graduated 

by grade 

Cutpoint 

students 

dropped out 

graduate on time 

on time 

Attendance 


Attend 






> 90 percent 

5,116 

4 

17 

79 

Grade 8 

Attend 

< 90 percent 

1,002 

18 

37 

45 


Attend 






< 80 percent 

231 

33 

44 

23 


Attend 






> 90 percent 

4,853 

3 

14 

83 

Grade 9 

Attend 

< 90 percent 

1,265 

20 

42 

38 


Attend 






< 80 percent 

473 

32 

51 

17 

GPA 


GPA > 3.5 

2,340 

1 

7 

92 

Grade 8 

GPA < 3.0 

2,642 

13 

34 

53 


GPA < 2.0 

961 

21 

45 

34 


GPA > 3.5 

1,904 

1 

5 

94 

Grade 9 

GPA < 3.0 

3,170 

12 

33 

55 


GPA < 2.0 

1,569 

19 

46 

35 

GPA is grade point average. 





Note: Dropped out refers to students who dropped out within four years of entering grade 9. Stayed in school 
but did not graduate on time refers to students who did not drop out within four years of entering grade 9 but 

did not graduate within four years. 

Graduated on time refers to students who graduated within four years of 

entering grade 9. 






Source: Author’s analysis of data from Oregon state and district databases; see box 2 for details. 
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less than 80 percent and a GPA of less than 2.0 identified large proportions of students 
who did not graduate on time (see table 4). More specifically: 

• 77 percent of students with grade 8 attendance of less than 80 percent did not 
graduate on time. 

• 83 percent of students with grade 9 attendance of less than 80 percent did not 
graduate on time. 

• 66 percent of students with grade 8 GPA less than 2.0 did not graduate on time. 

• 65 percent of students with grade 9 GPA less than 2.0 did not graduate on time. 

Meanwhile, only 1-12 percent of students with attendance of less than 80 percent and a 
GPA of less than 2.0 graduated on time (table 5). 

In general, students identified by more indicators had lower on-time graduation rates than did 
students identified by fewer indicators. This means that students identified by more indicators 
had higher rates of dropping out or staying in school but not of graduating on time (table 6). 


Table 5. Percentages of students who dropped out, stayed in school but did not 
graduate on time, and graduated on time, identified by each indicator, grade 9 
cohort of 2007/08 


Indicator 

Percent of students identified by the 
indicator among students who... 

Dropped out 
(n = 403) 

Stayed in school 
but did not 
graduate on time 
(n = 1,216) 

Graduated on time 
(n = 4,499) 

Grade 8 attendance less than 80 percent 

19 

8 

1 

Grade 9 attendance less than 80 percent 

37 

20 

2 

Grade 8 grade point average less than 2.0 

51 

35 

7 

Grade 9 grade point average less than 2.0 

75 

60 

12 


Note: Dropped out refers to students who dropped out within four years of entering grade 9. Stayed in school 
but did not graduate on time refers to students who did not drop out within four years of entering grade 9 but 
also did not graduate within four years. Graduated on time refers to students who graduated within four years 
of entering grade 9. 

Source: Author's analysis of data from Oregon state and district databases; see box 2 for details. 


Table 6. Students identified by multiple indicators had higher rates of dropping out or not graduating 
on time, grade 9 cohort of 2007/08 


Percent stayed in 


Number of 
indicators 

Number of 
students 

Percent 
dropped out 

Rate 

ratio 3 

school but did not 
graduate on time 

Rate 

ratio 3 

Percent graduated 
on time 

Rate 

ratio 3 

4 

101 

46.5 

29.9 

46.5 

4.7 

6.9 

0.08 

3 

206 

37.4 

24.0 

51.9 

5.3 

10.7 

0.12 

2 

643 

15.9 

10.2 

54.0 

5.5 

30.2 

0.34 

1 

926 

12.0 

7.7 

31.7 

3.2 

56.0 

0.63 

0 

4,242 

1.6 

1.0 

9.9 

1.0 

88.6 

1.00 


Note: Dropped out refers to students who dropped out within four years of entering grade 9. Stayed in school but did not graduate on 
time refers to students who did not drop out within four years of entering grade 9 but did not graduate within four years. Graduated on 
time refers to students who graduated within four years of entering grade 9. 

a. Compares the percentages in each row within each column to the percentage in the last row (students identified by zero indicators). 
Source: Author's analysis of data from Oregon state and district databases; see box 2 for details. 
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Figure 4. More than 70 percent of students identified by one or more indicators did 
not graduate on time, grade 9 cohort of 2007/08 


Percent of students identified by one or more indicators 



Note: Figure displays the percent of students identified by one or more indicators for students with each grad- 
uation outcome. Dropped out refers to students who dropped out within four years of entering grade 9. Stayed 
in school but did not graduate on time refers to students who did not drop out within four years of entering 
grade 9 but did not graduate within four years. Graduated on time refers to students who graduated within four 
years of entering grade 9. 

Source: Author’s analysis of data from Oregon state and district databases; see box 2 for details. 


Overall, more than 70 percent of students who did not graduate on time were identified 
by one or more of the indicators, including 84 percent of students who dropped out and 
66 percent of students who stayed in school but did not graduate on time; 16 percent of 
students who graduated on time were identified by one or more indicators (figure 4)- 


Overall, more 
than 70 percent 
of students who 
did not graduate 
on time were 
identified by 
one or more of 
the indicators, 
including 
84 percent of 
students who 
dropped out and 
66 percent of 
students who 
stayed in school 
but did not 
graduate on time 


Implications of the study findings 


The study results have four implications for districts or others currently implementing or 
planning to implement an early warning system. 

First, the results suggest, as others have, that an early warning system might be based on 
students’ attendance and course achievement in grades 8 and 9 — information that is col- 
lected as part of normal recordkeeping and reporting. 

Second, the results call attention to weaker graduation outcomes for male students and 
for English learner students and a potential need for districts to explore interventions and 
strategies for these students to improve on-time graduation rates. 

Third, the thresholds of 80 percent attendance and a 2.0 GPA in grades 8 and 9 for iden- 
tifying students who did not graduate on time were developed specifically from a cohort 
of students in four Oregon Leadership Network districts, and they are a useful metric for 
those districts. Other districts may want to explore whether different thresholds are appro- 
priate for their context. 


12 


Finally, the four early warning indicators identified can focus the attention of parents, stu- 
dents, and community organizations in the four districts — and in other districts in the 
Northwest Region and nationally — on strategies for improving students’ attendance and 
course achievement in grades 8 and 9 to help ensure their on-time graduation from high 
school. 


Limitations of the study 


The study examined dropping out, staying in school but not graduating on time, and grad- 
uating on time for one cohort of students in four districts using demographic and program 
participation data together with a limited set of variables describing their attendance, 
achievement, and behavior in grades 8 and 9. The study did not examine the complete 
range of education experiences to which students were exposed within and outside school. 
Other factors at the student, school, or other levels that may improve predictions of stu- 
dents’ dropping out or graduating were not considered in the analysis. Furthermore, in this 
study proficiency on standardized tests did not predict dropout status after other variables 
were controlled for. Flowever, Oregon students do not have to pass a graduation test to 
earn a diploma. The predictive value of such tests could be different in states that require 
students to pass them to graduate. 

Districts participating in the study were not a random sample of Oregon districts. Con- 
sequently, the results may not generalize to other districts in the state. Furthermore, the 
study identified indicators that were strongly associated with not graduating within four 
years of entering grade 9, but the results do not support claims that low attendance or low 
achievement in grade 8 or 9 caused students not to graduate with their grade 9 cohort. 


The results call 
attention to 
weaker graduation 
outcomes for male 
students and for 
English learner 
students and a 
potential need for 
districts to explore 
interventions and 
strategies for 
these students to 
improve on-time 
graduation rates 


Students who transferred out of the study districts and whose graduation outcomes were 
subsequently not reported to the Oregon Department of Education were not included in 
the analyses. In addition, some students were missing test scores, attendance data, or GPAs 
and consequently were not included. Finally, an unknown number of students dropped out 
but were not identified as dropouts because of incomplete reporting of dropout status by 
districts to the state. This problem was more prevalent in Oregon for the grade 9 cohort 
of 2007/08 than for later cohorts because the Oregon Department of Education required 
more detailed reporting of dropout status for later cohorts of students. Whatever the 
sources of missing data, results of the study would be biased to the degree that the associa- 
tions between variables in the study and students’ high school outcomes were different for 
students with missing data than they were for students included in the study. 


Finally, the model was calibrated and implications were derived from the model based on 
one cohort of students in four districts. It is important to evaluate how well the relation- 
ships identified by the study hold for other cohorts in the same districts and for cohorts of 
students in other districts. 
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Appendix A. Study variables and 
characteristics of the student cohort in each district 


This appendix includes descriptions of the variables used in the study; information about 
the demographic characteristics, program participation, attendance, and achievement in 
grades 8 and 9; and history of suspension or expulsion in grade 9 for the student cohort in 
each district. 


Table Al. Variables used in the study 


Variable Description 


Graduation outcomes 


Dropped out 

This outcome refers to students who left school without earning a diploma or equivalent certificate and 
were not enrolled in a high school program in the United States, a postsecondary degree program, or 
a district-sponsored adult high school diploma program. Not counted as dropouts are students who 
left school early but earned a General Educational Development certificate (GED) or students who are 
deceased, schooled at home, placed in special facilities, or living outside the United States. Rules for 
identifying students as dropouts are identified in the Oregon Graduates and Dropouts Reporting Manual, 
published by the Oregon Department of Education (2009). Students who drop out are identified in the 
cumulative average daily membership collection that districts submit to the department (see table source 
below). The study identified students as dropouts if they were identified as dropouts in the Oregon 
Department of Education database within four years after they entered grade 9. 

Stayed in school but did 
not graduate on time 

This outcome refers to students who did not drop out within four years of entering grade 9 but did not 
graduate within four years. 

Graduated on time 

This outcome refers to students who received a diploma within four years of entering grade 9. Students 
earning a diploma are identified in the cumulative average daily membership collection that districts 
submit to the Oregon Department of Education. 


Demographic characteristics and program participation 


Gender 

Male or female as reported in the state enrollment file. 

Race/ethnicity 

This study reported race/ethnicity following Oregon Department of Education practice for statewide 
reporting. Categories are: American Indian/Alaskan Native, Asian/Pacific Islander, Black, Hispanic/ 
Latino, White, Multiracial/Multiethnic. Students identified with more than one race are reported as 
multiracial/multiethnic unless they were identified as Hispanic/Latino or the only two races were Asian 
and Pacific Islander. Asian or Pacific Islander students are reported as Asian/Pacific Islander. Students 
identified as Hispanic are reported as Hispanic/Latino even if multiple races were identified. 

Special education status 

Students eligible for Oregon special education with an individualized education program. 

English learner student 
status 

Students identified as having limited proficiency in English. English learner student status does not 
require that a student be enrolled in a program for English as a second language. 


Attendance, achievement, and behavior in grades 8 and 9 


Attendance rate 

The number of days a student was reported present divided by the number of school days. Attendance 
rate was calculated separately for grades 8 and 9. 

Grade point average 

Two grade point average measures were used: cumulative grade point average across all grade 8 courses 
and cumulative grade point average across all grade 9 courses. 

Achievement on state 
test 

Meeting or not meeting the standard for proficiency on the state tests in reading and math at grade 8. 

Behavior 

Whether a student was suspended or expelled in grade 9. 


Note: Sources of missing data are discussed in the Limitations of the study section. Results in the report are based on cases with 
complete information. 

Source: Adapted by the author from Oregon Department of Education (2010). 
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Table A2. High school outcomes by district, student demographic characteristics, and program 
participation, grade 9 cohort of 2007/08 






Percent stayed in 

Percent 



Percent of 

Percent 

school but did not 

graduated 

District and characteristic 

Number 

district total 

dropped out 

graduate on time 

on time 

District A 

1,173 

100 

7 

20 

73 

Male 

618 

53 

9 

23 

68 

Female 

555 

47 

4 

17 

78 

American Indian 

a 

a 

a 

a 

a 

Asian 

106 

9 

0 

10 

90 

Black 

a 

a 

a 

a 

71 

Flispanic 

129 

n 

12 

31 

57 

More than one race/other 

41 

4 

a 

a 

76 

White 

864 

74 

7 

20 

73 

In special education 

82 

7 

12 

33 

55 

Not in special education 

1,091 

93 

6 

19 

74 

English learner student 

144 

12 

13 

27 

60 

Non-English learner student 

1,029 

88 

6 

19 

75 

Suspended or expelled in grade 9 

127 

11 

19 

45 

36 

Not suspended or expelled in grade 9 

1046 

89 

5 

17 

78 

District B 

1,750 

100 

6 

20 

73 

Male 

889 

51 

9 

23 

68 

Female 

861 

49 

4 

17 

79 

American Indian 

36 

2 

22 

25 

53 

Asian 

185 

11 

4 

12 

84 

Black 

186 

11 

10 

36 

54 

Flispanic 

186 

11 

15 

24 

62 

White 

1,127 

64 

4 

18 

78 

In special education 

36 

2 

a 

a 

47 

Not in special education 

1,714 

98 

6 

20 

74 

English learner student 

110 

6 

10 

23 

67 

Non-English learner student 

1,640 

94 

6 

20 

74 

Suspended or expelled in grade 9 

135 

8 

22 

49 

29 

Not suspended or expelled in grade 9 

1615 

92 

5 

18 

77 

District C 

599 

100 

12 

29 

59 

Male 

329 

55 

12 

34 

54 

Female 

270 

45 

12 

23 

65 

American Indian 

a 

a 

a 

a 

a 

Asian 

54 

9 

a 

a 

74 

Black 

45 

8 

a 

a 

60 

Flispanic 

186 

31 

19 

40 

41 

More than one race/other 

a 

a 

a 

a 

a 

White 

302 

50 

10 

23 

68 

In special education 

18 

3 

a 

a 

a 

Not special education 

581 

97 

12 

29 

59 

English learner student 

100 

17 

27 

37 

36 

Non-English learner student 

499 

83 

9 

27 

64 

Suspended or expelled in grade 9 

121 

20 

26 

51 

22 

Not suspended or expelled in grade 9 

478 

80 

9 

23 

68 


(continued) 
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Table A2. Demographic characteristics, program participation, and high school outcomes of students 
in the grade 9 cohort of 2007/08, by district (continued) 


District and characteristic 

Number 

Percent of 
district total 

Percent 
dropped out 

Percent stayed in 
school but did not 
graduate on time 

Percent 
graduated 
on time 

District D 

2,596 

100 

5 

17 

77 

Male 

1,323 

51 

6 

20 

74 

Female 

1,273 

49 

4 

15 

80 

American Indian 

10 

<1 

a 

a 

a 

Asian 

340 

13 

a 

a 

87 

Black 

70 

3 

a 

a 

67 

Hispanic 

407 

16 

n 

31 

58 

More than one race/other 

148 

6 

10 

16 

74 

White 

1,621 

62 

4 

15 

81 

In special education 

42 

2 

a 

a 

62 

Not special education 

2,554 

98 

5 

17 

77 

English learner student 

260 

10 

12 

29 

60 

Non-English learner student 

2,336 

90 

5 

16 

79 

Suspended or expelled in grade 9 

181 

7 

18 

41 

41 

Not suspended or expelled in grade 9 

2,415 

93 

4 

16 

80 


Note: Dropped out refers to students who dropped out within four years of entering grade 9. Stayed in school but did not graduate on 
time refers to students who did not drop out within four years of entering grade 9 but did not graduate within four years. Graduated on 
time refers to students who graduated within four years of entering grade 9. Percentages may not sum to 100 because of rounding. 

a. Values are not shown when data refer to fewer than 10 students. 

Source: Author’s analysis of data from Oregon state and district databases; see box 2 for details. 
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Table A3. Average attendance, achievement, and behavior in grades 8 and 9 by 
graduation outcome and district, grade 9 cohort of 2007/08 (percent unless 
otherwise indicated) 


District, grade, and indicator 

Dropped out 

Stayed in school but did 
not graduate on time 

Graduated 
on time 

District A (number) 

80 

236 

857 

Grade 8 

Attendance 

88 

89 

95 

Grade point average 

2.0 

2.2 

3.2 

Proficient on state reading test 

35 

53 

78 

Proficient on state math test 

59 

57 

81 

Grade 9 

Attendance 

84 

88 

95 

Grade point average 

1.3 

1.7 

2.9 

Suspended or expelled 

30 

24 

5 

District B (number) 

112 

355 

1,283 

Grade 8 

Attendance 

87 

91 

95 

Grade point average 

2.0 

2.4 

3.3 

Proficient on state reading test 

58 

67 

87 

Proficient on state math test 

52 

67 

89 

Grade 9 

Attendance 

77 

85 

95 

Grade point average 

1.3 

1.7 

3.1 

Suspended or expelled 

27 

19 

3 

District C (number) 

73 

173 

353 

Grade 8 

Attendance 

90 

94 

96 

Grade point average 

2.2 

2.5 

3.3 

Proficient on state reading test 

27 

46 

72 

Proficient on state math test 

33 

53 

72 

Grade 9 

Attendance 

84 

90 

96 

Grade point average 

1.5 

1.8 

3.1 

Suspended or expelled 

44 

36 

8 

District D (number) 

138 

452 

2,006 

Grade 8 

Attendance 

86 

91 

95 

Grade point average 

1.8 

2.3 

3.2 

Proficient on state reading test 

51 

63 

80 

Proficient on state math test 

54 

68 

87 

Grade 9 

Attendance 

78 

88 

95 

Grade point average 

1.3 

1.8 

3.1 

Suspended or expelled 

23 

16 

4 


Note: Dropped out refers to students who dropped out within four years of entering grade 9. Stayed in school 
but did not graduate on time refers to students who did not drop out within four years of entering grade 9 but 
did not graduate within four years. Graduated on time refers to students who graduated within four years of 
entering grade 9. State tests in reading and math were only at grade 8. Suspension and expulsion data were 
not available for grade 8. 

Source: Author’s analysis of data from Oregon state and district databases; see box 2 for details. 
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Appendix B. Connections to previous research 


A large body of research has investigated characteristics of middle school students and 
students in grade 9 that provided early warning of students who eventually dropped out of 
high school. Findings from representative studies are summarized in this appendix. 

Characteristics of high school students associated with graduating or dropping out 

A 2005 study of Chicago students showed that students who had not accumulated five 
credits by the end of grade 9 and had received two or more Fs in a core subject graduated 
on time at lower rates than grade 9 students who achieved at higher levels (Allensworth & 
Easton, 2005). Chicago students characterized by one or both of these factors were consul- 
ered to be “off track” and thus at risk of dropping out. Off-track rates varied greatly across 
schools, from 30 percent to 90 percent, which suggests the importance of monitoring and 
reporting students’ status as on track or off track by school. A later study of Chicago stu- 
dents found that grade 9 grade point average (GPA), whether a student was on track in 
grade 9, and the number of semester courses failed in grade 9 correctly identified graduates 
and nongraduates, while the number of absences was slightly less predictive of students 
who graduated (Allensworth & Easton, 2007). 

In Chicago, students’ achievement in grade 9 courses explained greater variance in their 
predicted graduation status than did their race/ethnicity, gender, economic status, or scores 
on a standardized test at grade 8 (Allensworth & Easton, 2007). The relative inefficien- 
cy of scores on standardized tests in predicting graduation implies that the transition 
to high school places more demands on students than simply acquiring academic skills 
(Allensworth & Easton, 2005). Other studies also demonstrate the relative inefficiency of 
scores on standardized tests compared with course performance and attendance in predict- 
ing graduation (Rumberger & Lim, 2008). 

Neild and Balfanz (2006) studied the progress of first-time grade 9 students in six gradu- 
ating classes of Philadelphia students from 2000 through 2005. Indicators and thresholds 
for attendance and achievement in grades 8 and 9 were identified that accurately predict- 
ed students who did not graduate from high school on time. Building on this research, a 
2010 study in seven Tennessee school districts with high dropout rates identified atten- 
dance, course failure, and suspension or expulsion in grade 9 as key factors associated with 
students graduating or not graduating (Balfanz, Wang, & Byrnes, 2010). These factors 
proved to be stronger predictors of dropping out than students’ demographic character- 
istics and test scores. In the seven districts, half the students who dropped out had less 
than 85 percent attendance and two or more course failures, and a third had two or more 
suspensions. These factors were not highly correlated, meaning their ability to predict 
whether a student would graduate or drop out was higher in combination than in isolation: 
24 percent of students with one indicator dropped out, while 43 percent of students with 
three indicators dropped out. 

Similarly, a 2010 study that used statewide data for Delaware students in grades 9-12 
during the 2006/07 and 2007/08 school years identified attendance and achievement indi- 
cators and thresholds that were highly predictive of whether a student would graduate on 
time (Uekawa, Merola, Fernandez, & Porowski, 2010). 
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Characteristics of middle school students associated with dropping out of high school 

A study of Philadelphia public school students yielded predictors of graduation similar 
to those identified with Chicago high school students, but for younger students (Balfanz, 
Herzog, & Mac Iver, 2007). For students in Philadelphia middle schools, poor attendance, 
poor behavior, or failing math or English in grade 6 was strongly associated with failing to 
graduate. Similar to the results in Chicago, scores on standardized tests were less predictive 
of graduation status than grades, attendance, and behavior were. 

Balfanz (2009) showed that the findings from Philadelphia were largely consistent with 
those from five other districts but noted differences among districts in the thresholds for 
signaling danger of not graduating. For example, in some districts missing a month or more 
of classes in middle school signaled that a student was in danger of not graduating, while 
in other districts missing at least two months of school signaled that a student was in 
danger of not graduating. Similarly, in some districts failing middle school math or English 
predicted being off track to graduate from high school, while in other districts failing any 
course predicted being off track. Balfanz also noted that on-time graduation rates varied 
across middle schools in the five districts (as low as 50 percent in some). In addition, 
schools with demographically similar students had different graduation rates, suggesting 
that schools can make a difference in keeping students on track toward graduation. 

Overall, the research literature highlights a number of potential early indicators of gradu- 
ation and thresholds that trigger identification of students at risk of not graduating. These 
are summarized in table Bl. 
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Table Bl. Grade 8 and 9 indicators and thresholds from previous research identifying students who 
dropped out of high school 


1 Study 

Indicators 

Thresholds !j 

Allensworth & Easton (2005) 

Number of (full-year) credits earned 

Fewer than five 


Number of (semester) Fs in core subjects 

At most one 

Allensworth & Easton (2007) a 

Attendance 

10 days per semester 


Grade point average and course Fs 

More than one F and grade point average 
in passed courses below 2.0 

Balfanz etal. (2010) b 

Attendance 

Less than 85 percent 


Course failures 

Two or more 


Suspensions 

Two or more 

Neild & Balfanz (2006)° 

Attendance, grade 8 

Less than 80 percent 


Grades in math and English, grade 8 

F in English and/or math 


Attendance 

Less than 70 percent 


Grades in math and English 

Fewer than two credits 


Promotion to grade 10 

Not promoted on time 

Uekawa et al. (2010) d 

Attendance 

Less than 88 percent 


Current grade in math 

Less than -0.47 


Current grade in English language arts 

Less than -0.63 


Note: Indicators and thresholds are for grade 9 except where identified for grade 8 and for Uekawa et al. (2010). 

a. Results extended from Allensworth and Easton (2005). Allensworth and Easton (2007) combined the two indicators and thresh- 
olds to form an “on-track" indicator: 78 percent of students identified as not on track failed to graduate in four years. Thresholds for 
grade point average for Allensworth and Easton (2005) were the values in figure 9 of their paper at which approximately 50 percent of 
students failed to graduate. Thresholds for attendance were the number of missed days in figure 12 of their paper at which students 
accumulated two Fs. 

b. Indicators and thresholds were selected to balance the need for accuracy (as shown by the large fraction of students above the 
threshold who dropped out) and yield (as shown by the threshold identifying a large fraction of all students who dropped out). 

c. Grade 8 indicators and thresholds were selected to identify students who dropped out at a rate of 85 percent or greater, and grade 9 
indicators and thresholds were selected to identify students with a 75 percent dropout rate. 

d. Thresholds for grades in math and English language arts are z-scores created by transforming course grades to a scale with mean 0 
and standard deviation 1. 

Source: Author's summary of cited studies. 
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Appendix C. Modeling graduation outcomes with logistic regression 


Logistic regression has been employed in a number of education studies similar to this one. 
For example, Gleason and Dynarski (2002) used logistic regression to examine whether 
widely used indicators of risk are effective predictors of students likely to drop out. Hierar- 
chical generalized linear modeling was applied in groundbreaking studies at the University 
of Chicago Consortium on Chicago School Research (Allensworth & Easton, 2007) on 
early identification of students likely to drop out of Chicago schools. Fong, Huang, and 
Goel (2008) used generalized linear modeling to examine links between math coursework 
in grade 12 and remediation in math for students entering higher education in Nevada. 
As a final example, Balfanz et al. (2010) applied generalized linear modeling to develop 
early warning indicators of students dropping out of Tennessee schools for the Tennessee 
Department of Education. 

Model-building strategy 

Previous research identified a number of variables concerning students’ experiences in 
grades 8 and 9 that identified students who later dropped out of high school (see table B1 
in appendix B). These include attendance in grades 8 and 9, suspensions in grade 9, and 
measures of achievement in grades 8 and 9 (grade point average [GPA] in grade 9, credits 
earned in grade 9, Fs in grade 9 core courses, and poor grades in English and math in 
grade 8). 

The base model for this study included students’ gender, race/ethnicity, special educa- 
tion status, English learner student status, and suspension or expulsion in grade 9. It also 
included some — but not all — of the variables identified in other studies as early warning 
indicators of students who dropped out. More specifically, the model included attendance 
in grades 8 and 9 and GPA in grades 8 and 9. It also included achievement on state reading 
and math tests at grade 8 to test the contribution from this source after controlling for 
other variables, including GPA. Credits earned and Fs in core subjects were not included 
because the districts reported these variables on different schedules (such as semester or 
trimester). Furthermore, for the cohort studied, districts did not report GPA and course 
achievement data directly to the Oregon Department of Education in a standard format. 
This meant that the study team had to use GPA data derived from the districts. Because 
these data were not reported in a consistent way, it was difficult to develop a measure 
of credits earned and Fs in core courses that was consistent across the districts. For this 
reason, these variables were not included in the base model. It is, of course, unclear 
whether — even if reported in a standard format — statewide credits earned and number 
of Fs in core courses would have added anything to explaining high school outcomes that 
was not already accounted for by GPA. 

The relationship of the predictors with students’ high school outcome was analyzed 
using ordered logistic regression. “Ordered” refers to the fact that the three graduation 
outcomes — dropping out within four years of entering grade 9, staying in school but not 
graduating within four years of entering grade 9, and graduating within four years of 
entering grade 9 — have an intuitive “worse-to-better” ordering. Dropping out is the least 
desirable outcome, staying in school but not graduating on time is more desirable than 
dropping out, and graduating on time is the most desirable outcome. The scale is ordinal 
and not interval because the amount of “desirability” may differ between the outcomes. 
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The model tests the association between students’ high school graduation outcomes 
and their gender, race/ethnicity, special education status, English learner student status, 
attendance, course achievement, scores on state tests, and suspension or expulsion. The 
model tests the strength of the association of each of these variables with graduation out- 
comes after controlling for the other variables. The model used in this study is the most 
popular model for ordinal responses and is based on logits of cumulative probabilities. For 
a response variable Y with J categories and a set of predictors X with corresponding coeffi- 
cients (2, the model has the form: 

logit[P (Y<=j | X)] = a r pXj=l ...J-l. 

The parameters [a] are called cutpoints; these quantities are estimated from the data to 
define the changes among the outcome categories but are rarely independently interpreta- 
ble. The model assumes that the odds ratios for the effects of the predictors on the response 
variable are the same for each of the possible ways of collapsing a /-category response to 
a binary variable. This model, which assumes that the effect (2 is the same for j, is thus 
referred to as the “proportional odds model” (Liu & Agresti, 2005). 

The regression software estimates the coefficients (2 together with the cutpoints k p k 2 , ..., 
k k v The coefficient estimates quantify the strength and direction of the association of 
each variable with the outcome after controlling for the effects of the other variables. 
Logistic regression coefficients are on the log odds scale. When they are exponentiated, 
the coefficients have greater intuitive meaning as odds ratios. For example, for a coefficient 
(3j of 2.1 for variable X p exp(^) = 2.1 means that the odds that an outcome occurs become 
2.1 times more likely when the value of X : increases by one unit. 

The model accounts for the clustering of students within schools by including fixed effects 
for schools. A fixed effect for a school is simply a variable coded 1 if a student attended the 
school and 0 otherwise. This simple construction has profound implications for estimates 
produced by the model. Models without school fixed effects produce estimates that combine 
what is happening both within and between schools. Including fixed effects for schools in 
the model produces estimates based only on students’ experiences within their school. 

Regression results 

Regression results are presented in table Cl. The strength and direction of the association 
of each variable with graduating on time compared with dropping out or not graduat- 
ing on time after controlling for the other variables are summarized in the odds ratio for 
the variable. Odds ratios greater than 1.0 indicate that higher levels of the variable are 
associated with graduating on time rather than dropping out or staying in school but not 
graduating on time, while odds ratios below 1.0 indicate that a higher level of the variable 
is associated less with graduating on time and more with dropping out or staying in school 
but not graduating on time. For example, the odds ratio of 0.77 for gender indicates that 
after the other variables are adjusted for, male students (coded 1) were less likely to grad- 
uate on time than female students (coded 0). The p-value indicates the level of statistical 
significance associated with each odds ratio. The level conventionally chosen for deciding 
whether an effect is statistically significant is 0.05. The p-value for gender indicates that 
lower rate of graduating on time for male students compared with female students is sta- 
tistically significant beyond the .05 level; in fact, it is significant beyond .001. Finally, for 
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Table Cl. Results of regression analysis of graduation outcome, grade 9 cohort of 
2009/08 


Variable 

Odds ratio 

P>|Z| 

95 percent confidence interval 

Demographic characteristics 

Gender (male) 

0.77 

.000 

0.67 

0.89 

Race 8 

American Indian 

0.85 

.576 

0.48 

1.50 

Asian 

0.89 

.411 

0.68 

1.17 

Black 

0.84 

.233 

0.64 

1.12 

Hispanic 

1.00 

.984 

0.82 

1.23 

More than one race/other 

0.77 

.128 

0.54 

1.08 

Special education status 

In special education 

0.89 

.522 

0.64 

1.26 

English learner student status 

English learner student 

0.68 

.001 

0.54 

0.86 

Attendance 

Grade 8 

1.02 

.001 

1.01 

1.03 

Grade 9 

1.03 

.000 

1.02 

1.03 

Grade point average 

Grade 8 

1.24 

.000 

1.11 

1.37 

Grade 9 

2.65 

.000 

2.39 

2.93 

Achievement on state test b 

Met reading standard 

1.13 

.142 

0.96 

1.33 

Met math standard 

1.04 

.663 

0.88 

1.23 

Behavior 

Suspended or expelled in grade 9 C 

0.89 

.276 

0.73 

1.09 


Note: Number of observations = 6,114. Likelihood ratio chi-square (56) = 2,499.2. Prob > chi-square = 
0.0000. Pseudo R 2 (McKelvey-Zavoina) = 0.46. Long (1997) recommends McKelvey-Zavoina as the most ap- 
propriate pseudo R 2 for ordinal models. The outcome variable for each student has three categories: dropped 
out, stayed in school but did not graduate on time, and graduated on time. Dropped out refers to students 
who dropped out within four years of entering grade 9. Stayed in school but did not graduate on time refers 
to students who did not drop out within four years of entering grade 9 but did not graduate within four years. 
Graduated on time refers to students who graduated within four years of entering grade 9. The model esti- 
mates rates of graduating on time relative to the other outcomes. 

a. White students are the base for the coefficients for the racial/ethnic categories. A joint significance test for 
the race variables was not statistically significant (likelihood ratio chi-square = 4.22 on 5 degrees of freedom, 
with p > .52). 

b. State test was only at grade 8. When the 0/1 proficiency indicators were replaced with scale scores, the 
resulting variables were not statistically significant and the fit of the model was unchanged. 

c. Data were not available for grade 8. 

Source: Author’s analysis of data from Oregon state and district databases; see box 2 for details. 


each variable the confidence interval for the estimated coefficient gives the range of values 
that is likely to capture the true — but unknown — size of the effect of the variable with 
95 percent confidence. The 95 percent confidence level corresponds to testing hypotheses 
using the conventional cutoff of p less than .05. 

Estimating the contribution of indicators to prediction 

The base model included gender, race/ethnicity, special education status, English learner 
student status, attendance in grade 9, GPA in grade 9, and suspension or expulsion in 


C-3 










grade 9. This served as the grade 9 “main effects” model. These variables were chosen 
because the alliance requested an investigation of how students’ background characteris- 
tics are associated with graduating and dropping out and because research shows that stu- 
dents’ attendance, achievement, and behavior in grade 9 significantly predict graduating 
and dropping out. Research also suggests that students’ experiences in middle school may 
be associated with their graduation outcomes. The alliance requested an exploration of 
this association in its data, thus students’ grade 8 scores on the state test and their atten- 
dance and behavior in grade 8 were included as candidate predictors. 

Collinearity was examined among the quantitative predictors: attendance in grade 8, 
attendance in grade 9, GPA in grade 8, and GPA in grade 9. Collinearity occurs when 
two or more predictors in a model are approximately determined by a combination of 
other independent predictors in the model. In severe cases of collinearity, the information 
carried by the collinear variables is largely redundant; very large standard errors can be a 
sign of this. No unusually large standard errors were identified. Nevertheless, two statistics 
— the tolerance and the variance inflation factor — were examined for each of the quanti- 
tative predictors. For each predictor the tolerance was close to 0 and the variance inflation 
factor was small. In short, no evidence of collinearity was found among the attendance 
and GPA variables. 

Assessing model fit 

The fit of the model was evaluated in three ways. First, the likelihood ratio chi-square 
test (see table Cl) was statistically significant, which indicates that at least one of the 
estimated coefficients was not zero. Second, the McKelvey-Zavoina pseudo R 2 of 0.46 was 
relatively large (McKelvey-Zavoina pseudo R 2 is perhaps the closest analogue to the famil- 
iar ordinary least squares R 2 among the various formulations of pseudo R 2 ). Third, ordinal 
regression assumes that the relationship between the predictors and the logits is the same 
across all outcomes (the proportional odds assumption described above). This assumption 
was tested by estimating coefficients for the predictors separately for each outcome and 
then testing whether the resulting estimates were equal. The overall chi-square test for 
equality across outcomes was statistically significant, which indicates a departure from the 
proportionality assumption. This situation frequently occurs with models based on logis- 
tic regression. Flowever, the differences were confined to 3 of the 10 predictors (the fixed 
effects for schools were not counted as predictors). As a result, no modifications were made 
to the model. 
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Notes 


1. Single-year dropout rates have been declining in Oregon for a number of years. The 
dropout rate in 1994/95 was 7.4 percent. Nationally, status dropout rates (that is, the 
percentage of individuals ages 16-24 who are not in school and have not earned a high 
school diploma or General Education Development certificate) declined between 1990 
and 2010 for White, Black, and Hispanic students (Aud et al., 2012). 

2. The statistical model also controlled for the schools that students attended. See appen- 
dix C for more information about the modeling used in the study. 
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